Playback of flash content at a client by redirecting execution of a script by a flash redirection plugin at a server to a flash redirection browser at the client

ABSTRACT

In one embodiment, a method includes, at a server, loading in a browser a webpage including flash content. The browser loads a flash redirection plugin, and requests the plugin to execute an ActionScript. The plugin passes the ActionScript to a flash redirection browser at a client. The plugin receives from the flash redirection browser the result of executing the ActionScript, and the result is provided to the browser.

PRIORITY

This application claims the benefit under 35 U.S.C. §119(a) of IndianPatent Application No. 3064/DEL/2012, filed 28 Sep. 2012, which isincorporated herein by reference. This application is acontinuation-in-part under 35 U.S.C. §120 of U.S. patent applicationSer. No. 13/539,226, filed 29 Jun. 2012, which is incorporated herein byreference. This application is a continuation-in-part under 35 U.S.C.§120 of U.S. patent application Ser. No. 13/539,224, filed 29 Jun. 2012,which is incorporated herein by reference. This application is acontinuation-in-part under 35 U.S.C. §120 of U.S. patent applicationSer. No. 13/539,222, filed 29 Jun. 2012, which is incorporated herein byreference. This application is a continuation-in-part under 35 U.S.C.§120 of U.S. patent application Ser. No. 13/539,220, filed 29 Jun. 2012,which is incorporated herein by reference.

FIELD

The disclosure relates in general to client-server computer systems, andmore particularly to presenting flash content.

BACKGROUND

A user of a client computing device may initiate a remote desktopprotocol (RDP) session with a server via any remote desktop protocolclient application executing on the client computing device (forexample, Microsoft Remote Desktop Protocol, Citrix Independent ComputingArchitecture, VMware Personal Computer over Internet Protocol, etc.).The user may attempt to load a webpage that includes flash content(e.g., a flash video) via a web browser executing on the server. Inresponse, the flash content of the webpage may be loaded on the serverand transmitted as a bitmap to the client computing device, allowing theuser to see an image associated with the webpage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram illustrating an example of a first system forpresenting flash content.

FIGS. 1B-1C illustrate examples of components of the first system forpresenting flash content of FIG. 1A.

FIG. 2A is a block diagram illustrating an example of a second systemfor presenting flash content.

FIG. 2B illustrates examples of components of the second system forpresenting flash content of FIG. 2A.

FIG. 3 is a block diagram illustrating an example of the server of FIG.1A or 2A.

FIG. 4 is a block diagram illustrating an example of the clientcomputing device of FIG. 1A or 2A.

FIG. 5 is a conceptual block diagram illustrating an example of acomputing device.

FIG. 6 illustrates a simplified diagram of an example of a networksystem.

FIG. 7 is a conceptual block diagram of a local device virtualizationsystem.

FIGS. 8A-8H are a flow chart illustrating an example of an operation ofthe server.

FIGS. 9A-9C are flow charts illustrating examples of operations of theserver.

DETAILED DESCRIPTION

The detailed description set forth below is intended as a description ofvarious configurations of the subject technology and is not intended torepresent the only configurations in which the subject technology may bepracticed. The appended drawings are incorporated herein and constitutea part of the detailed description. The detailed description includesspecific details for the purpose of providing a thorough understandingof the subject technology. However, it will be apparent to those skilledin the art that the subject technology may be practiced without thesespecific details. In some instances, well-known structures andcomponents are shown in block diagram form in order to avoid obscuringthe concepts of the subject technology. Like components are labeled withidentical element numbers for ease of understanding.

A user of a client computing device may initiate a remote desktopprotocol session with a server via a remote desktop protocol clientapplication executing on the client computing device. The remote desktopprotocol may be, for example, Microsoft Remote Desktop Protocol® (RDP),Citrix Independent Computing Architecture® (ICA), VMware VMview, VMwareRemote Desktop Protocol, or VMware Personal Computer Over InternetProtocol (PCOIP). The user may instantiate a web browser on the serverand data from the web browser may be displayed on the client computingdevice via the remote desktop protocol client application. The webbrowser may display a webpage that includes flash content, for example,a flash video that may be played via a flash player application (e.g.,Adobe Flash Player®).

In some examples, a computer program product may transmit to the clientcomputing device all flash content of a webpage or, alternatively,user-selected flash content of the webpage. The flash content may berendered at the server and transmitted to the client computing devicevia the remote desktop protocol. Alternatively, the computer programproduct may redirect the flash parameters (e.g., a web address oruniform resource locator of flash content) to the client computingdevice for downloading flash content at the client computing device viaan Internet connection of the client computing device and for renderingon the client computing device.

In some examples, the decision of flash content rendering may be manual.A compatibility list listing which flash content items are rendered onthe server and which content items are rendered on the client may beprovided. In some examples, a flash player may be required on the clientcomputing device. In other examples, the flash player may not berequired on the client computing device. In some examples, Internetconnectivity at the client computing device may be required. In otherexamples, Internet connectivity at the client computing device may notbe required. In some examples of the compatibility list approach, aserver (e.g., a login server) may fetch data from listed website(s) andpass the data back to the client, e.g., via a tunnel.

The present disclosure may, in particular embodiments, be supported inmultiple web browsers (e.g., Internet Explorer® versions 6, 7, 8, 9,and/or web browser implementing Netscape Plugin Application ProgrammingInterface (NPAPI), such as Chrome® and Mozilla Firefox®) and on multipleplatforms (e.g. Windows or Linux). Transparent flash content may besupported by the present disclosure in particular embodiments. Atransparent flash content item may include, for example, a flash videohaving a non-rectangular shape (e.g., a circular shape) or a flash videohaving hypertext markup language (HTML) content from a webpage in itsbackground (e.g., a video overlaying HTML images or text thatoccasionally becomes transparent, allowing the overlayed HTML images ortext to be viewed).

Particular embodiments include, but are not limited to: (1) flashredirection, (2) intelligent fallback to flash acceleration, (3) flashredirection via proxy server if the client computing device is notdirectly connected to the Internet but is connected to the Internet viaa proxy server, (4) flash redirection with a tunneling server (which maybe the same as the remote desktop protocol server) if the clientcomputing device does not have access to the Internet either directly orvia the proxy server, and/or (5) selective redirection of flash contentitem(s) on a webpage.

Particular embodiments provide one or more of the following: (1) flashredirection with flash acceleration, (2) flash redirection with proxy,(3) automatic tunneling of flash redirection, (4) selective redirectionof flash, (5) transparent flash support, (6) flash redirection supporton other browsers, for example, Mozilla Firefox® and Chrome®, and/or (7)automatic reloading of a webpage upon reconnecting to a disconnectedsession.

Particular embodiments may allow a user to play flash (e.g., AdobeFlash®) content at thin clients rather than playing at a server andconsuming server resources. Particular embodiments may allow users toexperience smoother and more reliable playback of flash content. Byplaying the flash content at the client, flash redirection may reducethe server and network load and may result in greater scalability whileimproving user experience. Flash content may include, for example,videos and animations. If the client supports flash redirection, theclient may, in particular embodiments, receive the original compressedmultimedia stream from the web server (e.g., flash content server) anddecode it locally for display. As a result, bandwidth usage may bedecreased as the data transmitted over the network may be compressedvideo instead of a succession of bitmaps. The bandwidth usage may remainthe same when a video window is resized or when played in full-screenmode. Flash redirection may work with Internet Explorer and NPAPIbrowsers like Chrome and Firefox. In particular embodiments, underseamless flash redirection, all the flash content of a page may beredirected to the client, as illustrated, for example, in FIG. 1B.

If a flash player is not present at the client, flash acceleration (see,e.g., FIG. 8B or 9B) may take place automatically for the flash contentat the server rather than remote desktop protocol (e.g., RDP, ICA, orVMVIEW) rendering. Flash acceleration may enable thin client users toexperience improved flash video content performance in a remotecomputing environment. In some examples, flash acceleration transcodesthe native flash media stream to a high bit rate motion JPG content thatis then delivered to the client for fast local rendering. Since theresulting “images” are “redirected” over a separate virtual channel tothe client, greater control of frame rate and compression may beprovided. In this example, a flash player need not be installed on theclient. A user of a remote desktop protocol client application may havean option, the settings for which may be stored in the server, to selectflash acceleration and/or flash redirection.

If a flash player is present at the client but the client does not haveaccess to the Internet, then flash redirection may happen through anexternal proxy server (e.g., proxy server 130A, 111C, or 121C). Theexternal proxy server may download the flash content and then send theflash content to the clients. Automatic detection of the proxy servermay happen at the client. If the proxy server is not present in thenetwork, then flash tunneling (see, e.g., FIG. 2A or 2B) may happenthrough the login server (e.g., remote desktop protocol server 120A or208B).

In particular embodiments, a user may selectively redirect the flash ona website at the server. As a result, the user may avoid redirecting theflash content items which are not of his/her interest, for example,advertisements. This may be achieved, for example, by the user selecting(e.g., single clicking via a mouse, double clicking via a mouse, ortouching via a touch screen) on the flash content item. This mechanismis called tapping. Once the user taps a particular flash content item ona website, the content items may be redirected. When the user opens thewebsite on a subsequent occasion, particular embodiments may detect thetapped flash content and start redirecting the flash to clients (e.g.,thin clients) automatically, such that the user does not need to tap thedesired flash content item(s) again. In some examples, by default allthe flash content items may be rendered at the server and the flashcontent item(s) may be redirected to client by tapping (see, e.g., FIG.8C or 9C).

Transparent flash allows the flash content to be in a shape other than arectangle, including, for example, a circle. In some implementations offlash redirection, transparent flash is not supported. In particularembodiments of flash redirection, transparent flash is supported, andall flash content is drawn in the same fashion as in a browser executinglocally on the client computing device.

In some examples, when a session is disconnected while redirecting theflash content and logged into again, all the flash content which wasplayed and redirected in the previous session may start redirecting tothe client automatically by reloading the webpage for a new session. Inparticular embodiments, when a user disconnects the session whileredirecting the flash and later resumes the session, the user may see ablank area at the locations of flash content on the webpage. The usermay see a white area at the server (in the client remote desktopprotocol application window representing the server) due to the flashredirection and the flash content being played locally at client. Adisconnect of a session may allow the server to retain all or a portionof the process(es) running on the server during the session.

In particular embodiments, when a web browser (e.g., Internet Explorer,Firefox, or Chrome) executing on the server loads a webpage which hasflash content, the flash content items on the webpage are monitored atthe server, and all the flash parameters are redirected from the serverto a client. The client may include a flash player and may pass all theparameters from the server to the flash player at the client. Somewebpages may include flash content that depends on the execution ofActionScript or JavaScript. As an example, a user viewing flash videocontent may wish to “pop out” the video embedded in a webpage to astandalone window and continue playing the video from where it left off;this may depend on the execution of JavaScript. As another example, aflash video player may be resized (e.g. YouTube® options to resize smallsize player to large size player or large size player to small sizeplayer) during viewing, and this may also depend on the execution ofJavaScript. In particular embodiments, the web browser executing on theserver may load a webpage which has flash content, and which depends onthe execution of an ActionScript. In such cases, the server may pass theActionScript to the client for remote execution at the client flashplayer, to be described in further detail below. In particularembodiments, the webpage may have flash content that depends on theexecution of a JavaScript. In such cases, the client may pass theJavaScript to the server for remote execution at the servers' browser,to be described in further detail below.

A flash redirection module (e.g., flash redirection client module 435)may check for the flash player at the client. If present, the flashredirection module may wait for flash parameters from the server. Ifthere are any parameters, the flash redirection module may start playingin a local flash player. If the flash player is not present at theclient, flash acceleration may be used instead of flash redirection.

Flash redirection at the client may include checking for access to anoutside network (e.g., the Internet). If the client has access to theoutside network, the client may fetch the data directly for the flashplayer. If the client does not have access to the outside network, theclient may look for an external proxy and forward all the requests ofits flash player to the proxy server. If an external proxy is not found,the client may query the server through tunneling. The tunneling server(which may be the same machine as the remote desktop protocol server)may fetch the data and pass back to the client. The tunneling server mayact like a pass through server. The tunneling server may forward theclient requests to the actual server (the remote desktop protocolserver) and pass back the response to client. If tunneling is used andthe server is over a wide area network (WAN), WAN Accelerator/Optimizermay be used to improve the performance.

In particular embodiments, the flash content on the webpage may bemonitored and a data repository may be prepared. Flash redirection maydisplay a small icon near flash content item(s) on the webpage. When theuser selects (e.g., clicks) on the icon, flash redirection may redirectall the associated flash parameters to the client. The client mayinclude a flash player and pass all the parameters from a server to theflash player.

Example of First System for Presenting Flash Content

FIG. 1A illustrates an example of a first system 100A for presentingflash content. As shown, the system 100A includes a client computingdevice 110A, a server 120A, a proxy server 130A, and a flash contentserver 150A. One or more of the client computing device 110A, the server120A, the proxy server 130A, and the flash content server 150A (e.g.,web server) may be connected to a network 140A (e.g., the Internet).While FIG. 1 includes only a single client computing device 110A, server120A, proxy server 130A, and flash content server 150A, particularembodiments may be implemented in conjunction with one or more clientcomputing devices, servers, proxy servers, or flash content servers. Insome aspects, particular embodiments may be implemented without one ormore of the above-listed components (e.g., without a proxy server 130A).

The client computing device 110A may be a laptop computer, a desktopcomputer, a mobile phone, a tablet computer, a personal digitalassistant (PDA), or any similar device. As illustrated, the clientcomputing device is connected to the server 120A via a connection 125A.The connection 125A may be a remote desktop protocol connection. Theclient computing device 110A may include a flash player (e.g., AdobeFlash Player®) or a remote desktop protocol client application. Oneexample of the client computing device 110A is described in detail inconjunction with FIG. 4.

A remote desktop protocol connection may, among other things, allow forvisual data from the server 120A to be displayed remotely on the clientcomputing device 110A. The client computing device 110A may be able tocause execution of software application(s) residing on the server 120A.A remote desktop connection may use a remote desktop protocol. Exampleremote desktop protocols include Microsoft Remote Desktop Protocol®(RDP), Citrix Independent Computing Architecture® (ICA), and VMwareVMview®.

In one example, the connection 125A between the client computing device110A and the server 120A is independent of the network 140A. Forexample, the network 140A may be the Internet, and the client computingdevice 110A may be connected to the server 120A via one or more of alocal network, a corporate intranet, a direct wired connection, or adirect wireless connection. Alternatively, the client computing device110A may be connected to the server 120A via a connection 125A (e.g., aremote desktop protocol connection) over the network 140A.

The server 120A may be a remote desktop protocol server. In someaspects, the server may be an enterprise computer (e.g., an officedesktop computer) that is configured to be accessed remotely (e.g., froman employee's home or a field office) via a remote desktop protocol.Alternatively, the server 120A may be implemented as a server farmincluding multiple machines, a single machine with a single processor,or a multi-processor machine. One example of the server 120A isdescribed in detail in conjunction with FIG. 3.

In some examples, the client computing device 110A may be connected tothe network 140A directly via a connection 135A. The connection 135A maybe a hypertext transfer protocol (HTTP) or hypertext transfer protocolsecure (HTTPS) connection. In some examples, the client computing device110A may be connected to the network 140A via the proxy server 130A. Theconnection 145A between the client computing device 110A and the proxyserver 130A may be a HTTP or HTTPS connection, and the connection 155Abetween the proxy server 130A and the network 140A may be a HTTP orHTTPS connection. In some aspects, the client computing device 110A maynot be connected to the network 140A except via the server 120A. Theconnection 115A of the server 120A to the network 140A may be a HTTPconnection.

The proxy server 130A may be configured to allow a computer (e.g., aclient computing device 110A) connecting to the proxy server 130A toconnect to the network 140A. The proxy server 130A may be configured tofilter out some content (e.g., webpages) to prevent users of computersconnected to the network 140A via the proxy server 130A from accessingthat content. For example, in an enterprise setting, an employer may usethe proxy server 130A to prevent its employees from accessing socialnetworking content, while allowing them to access other Internetcontent.

Particular embodiments may be implemented without the proxy server 130Aor without one or more of the connections 135A, 145A, and 155A.

The flash content server 150A may be, for example, a web server thatincludes flash content (e.g., a YouTube® server). Flash content mayinclude video files or transparent flash content items. A transparentflash content item may include, for example, a flash video having anon-rectangular shape (e.g., a circular shape) or a flash video havinghypertext markup language (HTML) content from a webpage in itsbackground (e.g., a video overlaying HTML images or text thatoccasionally becomes transparent, allowing the overlayed HTML images ortext to be viewed). The flash content server 150A is connected to thenetwork 140A via a connection 105A. The connection 105A may be a HTTP orHTTPS connection.

HTTP may be used, for example, to fetch flash content by the clientcomputing device 110A. HTTPS may be used, for example, to get flashcontent in a secure manner by the client computing device.

FIG. 1B illustrates a system 100B that includes examples of components(110B, 112B, 114B, 116B, 118B, 122B, 124B, 126B, 128B, 130B, 132B, 134B,136B, and 140B) of the first system 100A for presenting flash content ofFIG. 1A.

As shown in FIG. 1B, in one example embodiment, server 132B on serverside 130B is displaying, via a remote desktop protocol serverapplication, a webpage that includes a flash video 134B (or other flashcontent item) from flash content server 140B. The server 132B isconnected to clients 112B and 122B via a remote desktop protocolconnection 118B and 128B (e.g., RDP, ICA, or VMview). Each client 112Bor 122B on client side 110B is connected to the flash content server140B via a hypertext transfer protocol (HTTP) connection 116B or 126B.The clients 112B or 122B receive the webpage as a bitmap via the remotedesktop protocol connection 118B or 128B and the clients 112B or 122Breceive the flash video 114B or 124B via the HTTP connection 116B or126B to the flash content server 140B.

FIG. 1C illustrates a system 100C that includes examples of components(110C, 111C, 112C, 113C, 114C, 116C, 118C, 121C, 122C, 123C, 124C, 126C,128C, 130C, 132C, 134C, 136C, and 140C) of the first system 100A forpresenting flash content of FIG. 1A.

As shown in FIG. 1C, in one example embodiment, server 132C on serverside 130C is displaying, via a remote desktop protocol serverapplication, a webpage that includes a flash video 134C (or other flashcontent item) from flash content server 140C. The server 132C isconnected to clients 112C and 132C via a remote desktop protocolconnection 118C and 128C (e.g., RDP, ICA, or VMview). Each client 112Cor 122C on client side 110C is connected to a proxy server 111C or 121Cvia a HTTP connection 116C or 126C. Each proxy server 111C or 121C isconnected to the flash content server 140C via a HTTP connection 113C or123C. The clients 112C or 122C receive the webpage as a bitmap via theremote desktop protocol connection 118C or 128C and the clients 112C or122C receive the flash video 114C or 124C via the HTTP connections 113Cand 116C or 123C and 126C and over the proxy server 111C or 121C to theflash content server 140C.

Example of Second System for Presenting Flash Content

FIG. 2A is a block diagram illustrating an example of a second system200A for presenting flash content. As shown, the system 200A includesthe client computing device 110A, the server 120A, and the flash contentserver 150A.

As illustrated, the client computing device 110A includes a flash player210A. The flash player 210A may be configured to play flash content, forexample flash videos or transparent flash content items. The flashplayer 210A may be, for example, Adobe Flash Player®. Also, asillustrated, the client computing device 110A is not directly connectedto the network 140A. Instead, the client computing device 110A isconfigured to access the network via the connection 125A to the serverand the connection 115A of the server to the network.

The connection 125A between the client computing device 110A and theserver 120A may be a remote desktop protocol connection. The clientcomputing device 110A may transmit to the server 120A user interfacedata (e.g., mouse operations and keyboard clicks entered by a user ofthe client computing device) via the connection 125A. The server 120Amay transmit to the client computing device 110A a bitmap includingimage data to be displayed on a display unit (e.g., a screen) of theclient computing device 110A via the connection 125A.

As illustrated, the server 120A includes a tunneling module 220A. Thetunneling module may be configured to, in an event that the server 120Ais connected to the client computing device 110A via a connection 125A(e.g., a remote desktop protocol connection) and a web browser executingon the server 120A and displayed on the client computing device 110A viaa remote desktop protocol client application is accessing a flashcontent item, create a tunnel 230A. The tunnel 230A may be configured totransmit flash content from the server 120A to the client 110A as aflash content file, rather than as a bitmap. As a result, the clientcomputing device 110A may be able to play the flash content via theflash player 210A, improving the quality of the playback of the flashcontent (over the bitmap transmitted via connection 125A).

The flash content may be received at the server 120A from the flashcontent server 150A via the network 140A. The server 120A may beconnected with the flash content server 150A via connections 115A and105A.

FIG. 2B illustrates a system 200B that includes examples of components(202B, 204B, 206B, 208B, 210B, 212B, 214B, and 216B) of the secondsystem 200A for presenting flash content of FIG. 2A.

Example of Server

FIG. 3 is a block diagram illustrating an example of the server 120A.

As shown, the server 120A includes a central processing unit (CPU) 305,a network interface 310, and a memory 315. The CPU 305 may include oneor more processors configured to execute computer instructions that arestored in a computer-readable medium, for example, the memory 315. Thenetwork interface 310 may be configured to allow the server 120A totransmit and receive data in a network, e.g., network 140A of FIG. 1A.The network interface 310 may include one or more network interfacecards (NICs). The memory 315 stores data or instructions. The memory 315may be one or more of a cache unit, a storage unit, an internal memoryunit, or an external memory unit. As illustrated, the memory 315includes a server flash virtualization module 320, a browser 350, aflash player 355, and settings 360.

As illustrated, the server flash virtualization module 320 includes aprerequisite check module 325, a compression modules 330, the tunnelingmodule 220A, an acceleration module 335, a redirection module 340, and anetwork module 345. In particular embodiments, redirection module 340includes a flash redirection proxy plugin, which may support thefunction of a flash plugin at client computing device 110A that dependon scripts (e.g. JavaScript). The proxy plugin may, for example, work toredirect flash parameters from the browser of the server to the flashplayer of the client. An example of an operation of the server flashvirtualization module 320 and the components 325, 330, 220A, 335, 340,and 345 thereof is provided in conjunction with FIGS. 8A-8H.

The browser 350 is configured to provide data from webpages for displayat a client (e.g., client computing device 110A) via a remote desktopprotocol client application executing at the client. Some of thewebpages may include flash content. Example browsers that may correspondto browser 350 include Microsoft Internet Explorer®, Mozilla Firefox®,or Google Chrome Browser®.

The flash player 355 executing on the server 120A may be configured toprovide output based on flash content for display at the client via theremote desktop protocol client application executing at the client. Forexample, the flash player 355 may place image data based on flashcontent into a bitmap for transmission to the client.

The settings 360 include settings of the server 120A. As shown, thesettings 360 include seamless or selective flash redirection settings365. At any time, the seamless or selective flash redirection settings365 indicate either (a) seamless flash redirection or (b) selectiveflash redirection. Under seamless flash redirection, all flash contentfrom a webpage viewed in the browser 350 during a remote desktopprotocol session with a client is redirected to the client for playingin the flash player 210A of the client. Under selective flashredirection, flash content selected by a user of the client (e.g., viaoperation of a mouse on the client) is redirected to the client forplaying in the flash player 210A of the client. Under selective flashredirection, flash content that is not selected by the user is renderedlocally at the server 120A via operation of the flash player 355 of theserver and transmitted to the client within a bitmap via the remotedesktop protocol connection. Selective flash redirection is described indetail in conjunction with FIGS. 8C and 9C.

Example of Client Computing Device

FIG. 4 is a block diagram illustrating an example of the clientcomputing device 110A.

As shown, the client computing device 110A includes a central processingunit (CPU) 405, a network interface 410, and a memory 415. The CPU 405may include one or more processors configured to execute computerinstructions that are stored in a computer-readable medium, for example,the memory 415. The network interface 410 is configured to allow theclient computing device 110A to transmit and receive data in a network,e.g., network 140A of FIG. 1A. The network interface 410 may include oneor more network interface cards (NICs). The memory 415 stores data orinstructions. The memory 415 may be one or more of a cache unit, astorage unit, an internal memory unit, or an external memory unit. Asillustrated, the memory 415 includes a client flash virtualizationmodule 420 and a flash player 210A.

As illustrated, the client flash virtualization module 420 includes adecoding module 425, a network module 430, a flash redirection clientmodule 435, and a rendering module 440. In particular embodiments, flashredirection client module 435 includes a flash redirection proxybrowser, which may support the function of a browser at server 120A thatdepend on scripts (e.g. ActionScript). The proxy browser may, forexample, be a container for the flash player and may be independent ofany browser operating at the server and be independent of the platformof the client (e.g. Windows or Linux). An example of an operation of theclient flash virtualization module 420 and the components 425, 430, 435,and 440 thereof is provided in conjunction with FIGS. 8A-8H.

As illustrated, the memory 415 of the client computing device 110A alsoincludes a flash player 210A. The flash player 210A is configured toplay flash content received at the client computing device 110A, forexample, via the tunnel 230A, via the connection 135A to the network140A, or via the connection 145A to the proxy server 130A. The flashplayer 210A may be, for example, Adobe Flash Player® or any other flashplayer.

Example of Computing Device

FIG. 5 is a conceptual block diagram illustrating an example of acomputing device.

A computing device 500 may be, for example, a client computing device110A, a server 120A, a proxy server 130A, or a flash content server150A. A computing device may comprise one or more computing devices.

A computing device 500 may include a processor 502. The processor 502may include one or more processors. The processor 502 is capable ofcommunication with a receiver 506 and a transmitter 508 through a bus504 or other structures or devices. It should be understood thatcommunication means other than busses may be utilized with the disclosedconfigurations. The processor 502 may generate commands, messages,and/or other types of data to be provided to the transmitter 509 forcommunication. In addition, commands, messages, and/or other types ofdata may be received at the receiver 506, and processed by the processor502.

The processor 502 may operate in conjunction with a general-purposeprocessor or a specific-purpose processor for executing instructions andmay further include a machine-readable medium 519 for storing dataand/or instructions for software programs. The instructions, which maybe stored in a machine-readable medium 510 and/or 519, are executable bythe processor 502 to control and manage access to the various networks,as well as provide other communication and processing functions. Theinstructions may also include instructions executable by the processor502 for various user interface devices, such as a display 512 and akeypad 514. The processor 502 may include an input port 522 and anoutput port 524. Each of the input port 522 and the output port 524 mayinclude one or more ports. The input port 522 and the output port 524may be the same port (e.g., a bi-directional port) or may be differentports.

The processor 502 may be implemented using software, hardware, or acombination of both. By way of example, the processor 502 may beimplemented with one or more processors. A processor may be ageneral-purpose microprocessor, a microcontroller, a digital signalprocessor (DSP), an application specific integrated circuit (ASIC), afield programmable gate array (FPGA), a programmable logic device (PLD),a controller, a state machine, gated logic, discrete hardwarecomponents, and/or any other suitable device that may performcalculations or other manipulations of information. Those skilled in theart will recognize how best to implement the described functionality forthe processor 502.

Software shall be construed broadly to mean instructions, data, or anycombination thereof, whether referred to as software, firmware,middleware, microcode, hardware description language, or otherwise.Instructions may include code (e.g., in source code format, binary codeformat, executable code format, or any other suitable format of code).Instructions may be executable, for example, by a computing device(e.g., a client computing device, an HTTP server, a web server) or by aprocessing system (e.g., an operating system, an HTTP server, or a webserver). Instructions may be, for example, a computer program includingcode.

A machine-readable medium may be one or more machine-readable media. Amachine-readable medium (e.g., 510) may include storage external to anoperating system, such as a random access memory (RAM) 550, a flashmemory 530, a read only memory (ROM) 540, a programmable read-onlymemory (PROM), an erasable PROM (EPROM), registers, a hard disk, aremovable disk, a CD-ROM, a DVD, or any other suitable storage device. Amachine-readable medium 519 may also have a volatile memory and anon-volatile memory. The machine-readable medium 519 may be anon-transitory machine-readable medium. A non-transitorymachine-readable medium may include one or more volatile and/ornon-volatile memories. A machine-readable medium 519 may include storageintegrated into a processing system, such as might be the case with anapplication specific integrated circuit (ASIC). A memory may be amachine-readable medium (e.g., 510 or 519) or a part thereof.

According to one aspect of the disclosure, a machine-readable medium isa computer-readable medium encoded or stored with instructions and is acomputing element, which defines structural and functionalinterrelationships between the instructions and the rest of thecomputing device, which permit the instructions' functionality to berealized. In one aspect, a machine-readable medium is a non-transitorymachine-readable medium, a machine-readable storage medium, or anon-transitory machine-readable storage medium. In one aspect, anon-transitory machine-readable medium is a computer-readable medium, anon-transitory computer-readable medium, a computer-readable storagemedium, or a non-transitory computer-readable storage medium.

An interface 516 may be any type of interface and may reside between anyof the components shown in FIG. 5. An interface 516 may also be, forexample, an interface to the outside world (e.g., an Internet networkinterface). A transceiver block 507 may represent one or moretransceivers, and each transceiver may include a receiver 506 and atransmitter 509. A functionality implemented in a processor 502 may beimplemented in a portion of a receiver 506, a portion of a transmitter509, a portion of a machine-readable medium 510, a portion of a display512, a portion of a keypad 514, or a portion of an interface 516, andvice versa. In one aspect, a computing device may include only some orall of the elements shown in FIG. 5. A computing device may includeother elements not shown in FIG. 5. A computing device may include morethan one of the same elements.

Example of Network System

FIG. 6 illustrates a simplified diagram of an example of a computernetwork system in accordance with an aspect of the present disclosure.

A computer network system 600 may include one or more client computingdevices 602 (e.g., laptop computers, desktop computers, tablets, PDAs,mobile phones, etc.) in communication with one or more server computingdevices 604 (e.g., a server such as an HTTP server, a web server, anenterprise server, etc.) via a network 606. In one aspect, a servercomputing device 604 is configured to allow remote sessions (e.g.,remote desktop sessions) wherein users may access applications and fileson the server computing device 604 by logging onto the server computingdevice 604 from a client computing device 602. Such a connection may beestablished using any of several well-known techniques such as theremote desktop protocol (RDP) on a Windows-based server or thetechniques disclosed herein for a non-Windows-based server.

In one aspect, a client computing device 602 may be an end-usercomputing device, such as a laptop or desktop computer. The clientcomputing device 602 may correspond to the client computing device 110Aof FIG. 1A. In one aspect, a server computing device 604 may correspondto one or more of the server 120A, the proxy server 130A, or the flashcontent server 150A of FIG. 1A.

By way of illustration and not limitation, a client computing device 602may represent a computer, a mobile phone, a laptop computer, a tablet, athin computing device, a personal digital assistant (PDA), a portablecomputing device, a virtual machine, or a suitable device with aprocessor. In one example, a client computing device 602 is a smartphone (e.g., iPhone, Android phone, Blackberry, etc.). In certainconfigurations, a client computing device 602 may represent an audioplayer, a game console, a camera, a camcorder, an audio device, a videodevice, a multimedia device, or a device capable of supporting aconnection to a remote computing device. In an example, a clientcomputing device 602 is mobile. In another example, a client computingdevice 602 is a hand-held device. In another example, a client computingdevice 602 may be stationary. In one example, a client computing device602 may be a device having at least a processor and memory, where thetotal amount of memory of the client computing device 602 is less thanthe total amount of memory in a server computing device 604. In anexample, a client computing device 602 does not have a hard disk. In oneexample, a client computing device 602 has a display smaller than adisplay supported by a server computing device 604.

In one aspect, a server computing device 604 may represent a computer, alaptop computer, a computing device, a virtual machine (e.g., VMware®Virtual Machine), a desktop session (e.g., Microsoft Terminal Server), apublished application (e.g., Microsoft Terminal Server) or a suitabledevice with a processor. In one aspect, a server computing device 604may be stationary. In another aspect, a server computing device 604 maybe mobile. In certain configurations, a server computing device 604 maybe any device that may represent a computing device. In one aspect, aserver computing device 604 may include one or more computing devices.

In one example, a first device is remote to a second device when thefirst device is not directly connected to the second device. In oneexample, a first remote device may be connected to a second device overa communication network such as a Local Area Network (LAN), a Wide AreaNetwork (WAN), and/or other network.

When a client computing device 602 and a server computing device 604 areremote with respect to each other, a client computing device 602 mayconnect to a server computing device 604 over a network 606, forexample, via a modem connection, a LAN connection including the Ethernetor a broadband WAN connection including DSL, Cable, T1, T3, FiberOptics, Wi-Fi, or a mobile network connection including GSM, GPRS, 3G,WiMax or other network connection. A network 606 may be a LAN network, aWAN network, a wireless network, the Internet, an intranet or othernetwork. A remote device (e.g., a computing device) on a network may beaddressed by a corresponding network address, such as, but not limitedto, an Internet protocol (IP) address, an Internet name, a WindowsInternet name service (WINS) name, a domain name or other system name.These illustrate some examples as to how one device may be remote toanother device. However, the present disclosure is not limited to theseexamples.

Example Virtualization System

FIG. 7 is a block diagram of a local device virtualization system 700,according to an aspect of the disclosure. The system 700 may include theclient 702 in communication with the server 704, for example, over anetwork (as illustrated in detail in FIG. 6). The client 702 may includea proxy 710, a stub driver 720, and a bus driver 730. The client 702 maybe connected to a device 740, as shown in FIG. 7. The server 704 mayinclude an agent 750, and a virtual bus driver 760.

According to the illustrated configuration, while the device 740 is notlocally or physically connected to the server 704 and is remote to theserver 704, the device 740 appears to the server 704 as if it is locallyconnected to the server 704, as discussed further below. Thus, thedevice 740 appears to the server 704 as a virtual device 790.

By way of illustration and not limitation, the device 740 may be amachine-readable storage medium (e.g., flash storage device), a printer,a scanner, a camera, a facsimile machine, a phone, an audio device, avideo device, a peripheral device, or other suitable device that may beconnected to the client 702. The device 740 may be an external device(i.e., external to the client 702) or an internal device (i.e., internalto the client 702).

In one aspect of the disclosure, the device 740 is a Universal SerialBus (USB) device that may be locally connected to the client 702 using awired USB or wireless USB connection and communicates with the client702 according to a USB communications protocol. In another aspect, thedevice 740 may be a device other than a USB device.

As used herein, a “local” device of a system, or a device “locally”connected to a system, may be a device directly connected to the systemusing one or more wires or connectors (e.g., physically connected to thesystem), or a device directly connected to the system using a wirelesslink (e.g., Bluetooth). For example, device 740 is a local device ofclient 702. Furthermore, in one aspect of the disclosure, a local deviceof a system or a device locally connected to a system may include adevice within the system (e.g., an internal device of client 702).

A “remote” device, or a device “remote” to a system, may be a devicethat is not directly connected to the system. For example, the server704 is remote to both client 702 and device 740 because server 704 isnot directly connected to client 702 or device 740 but connectedindirectly through network 606 (illustrated in FIG. 6), which mayinclude, for example, another server, or the Internet.

The bus driver 730 may be configured to allow the operating system andprograms of the client 702 to interact with the device 740. In oneaspect, when the device 740 is connected to the client 702 (e.g.,plugged into a port of the client 702), the bus driver 730 may detectthe presence of the device 740 and read information regarding the device740 (“device information”) from the device 740. The device informationmay include features, characteristics and other information specific tothe device. For an example of a USB device, the device information maycomprise a device descriptor (e.g., product ID, vender ID and/or otherinformation), a configuration descriptor, an interface descriptor, anendpoint descriptor and/or a string descriptor. The bus driver 730 maycommunicate with the device 740 through a computer bus or other wired orwireless communications interface.

In one aspect, a program (e.g., application) running locally on theclient 702 may access the device 740. For example, the device 740 may beaccessed locally when the client 702 is not connected to the server 704.In this aspect, the operating system (e.g., Microsoft Windows®) of theclient 702 may use the device information to find and load anappropriate device driver (not shown) for the device 740. The devicedriver may provide the program with a high-level interface to the device740.

In one aspect, the device 740 may be accessed from the server 704 as ifthe device were connected locally to the server 740. For example, thedevice 740 may be accessible from the desktop running on the server 704(i.e., virtual desktop environment). In this aspect, the bus driver 730may be configured to load the stub driver 720 as the default driver forthe device 740. The stub driver 720 may be configured to report thepresence of the device 740 to the proxy 710 and to provide the deviceinformation (e.g., device descriptor) to the proxy 710.

The proxy 710 may be configured to report the presence of the device740, along with the device information, to the agent 750 of the server704 over the network 606 (illustrated in FIG. 6). Thus, the stub driver720 redirects the device 740 to the server 704 via the proxy 710.

The agent 750 may be configured to receive the report from the proxy 710that the device 740 is connected to the client 702 and the deviceinformation. The agent 750 may provide notification of the device 740,along with the device information, to the virtual bus driver 760. Thevirtual bus driver 760 may be configured to report to the operatingsystem of the server 704 that the device 740 is connected and to providethe device information to the operating system. This allows theoperating system of the server 704 to recognize the presence of thedevice 740 even though the device 740 is connected to the client 702.The operating system of the server 704 may use the device information tofind and load an appropriate device driver 780 for the device 740 at theserver 704, an example of which is illustrated in FIG. 7. As a result,the device 740 is enumerated on the server 704. Once the presence of thedevice 740 is reported to the operating system of the server 704, thedevice 740 may be accessible from the desktop running on the server 704(i.e., virtual desktop environment). For example, the device 740 mayappear as an icon on the virtual desktop environment and/or may beaccessed by applications running on the server 704.

In one aspect, an application 770 running on the server 704 may accessthe device 740 by sending a transaction request for the device 740 tothe virtual bus driver 760 either directly or through the device driver780. The virtual bus driver 760 may direct the transaction request tothe agent 750, which sends the transaction request to the proxy 710 overthe network 606. The proxy 710 receives the transaction request from theagent 750, and directs the received transaction request to the stubdriver 720. The stub driver 720 then directs the transaction request tothe device 740 through the bus driver 730.

The bus driver 730 receives the result of the transaction request fromthe device 740 and sends the result of the transaction request to thestub driver 720. The stub driver 720 directs the result of thetransaction request to the proxy 710, which sends the result of thetransaction request to the agent 750 over the network 606. The agent 750directs the result of the transaction request to the virtual bus driver760. The virtual bus driver 760 then directs the result of thetransaction request to the application 770 either directly or throughthe device driver 780.

Thus, the virtual bus driver 760 may receive transaction requests forthe device 740 from the application 770 and send results of thetransaction requests back to the application 770 (either directly orthrough the device driver 780). As such, the application 770 mayinteract with the virtual bus driver 760 in the same way as a bus driverfor a device that is connected locally to the server 704. The virtualbus driver 760 may hide the fact that it sends transaction requests tothe agent 750 and receives the results of the transaction requests fromthe agent 750 instead of a device that is connected locally to theserver 704. As a result, the device 740 connected to the client 702 mayappear to the application 770 as if the physical device 740 is connectedlocally to the server 704.

In one example, the client 702 correspond to the client computing device110A, and the server 704 corresponds to the server 120A. The device 740corresponds to a device (e.g., a mouse or a keyboard) of the clientcomputing device 110A, which is virtualized on the server 120A as thevirtual device 790. The connection between the client 702 and the server704 may correspond to the connection 125A (e.g., a remote desktopprotocol connection).

Example of an Operation of the Server for Presenting Flash Content

FIGS. 8A-8H are a flow chart illustrating an example process 800 of anoperation of the server for presenting flash content.

With reference to FIG. 8A, the process 800 begins at operation 802,where a remote desktop protocol server (e.g., server 120A) facilitatesdisplay of a webpage. The webpage may be displayed by operation of abrowser (e.g., browser 350) executing on the remote desktop protocolserver. The webpage may be displayed via a remote desktop protocolclient application executing on a client computing device (e.g., clientcomputing device 110A). The client computing device may receive, fromthe remote desktop protocol server, a bitmap that includes image dataassociated with the webpage via the remote desktop protocol connection.The client computing device may provide for display of the bitmap, forexample, via a display unit (e.g., a screen) of the client computingdevice. The webpage may include flash content (e.g., a single flashcontent item or multiple flash content items). The flash content item(s)may include one or more flash video(s) and/or one or more transparentflash content item(s), and the flash content items may interact withscripts including, for example, JavaScript or ActionScript. The remotedesktop protocol server and the client computing device may be connectedvia a remote desktop protocol connection (e.g., connection 125A). Theremote desktop protocol may be, for example, Microsoft Remote DesktopProtocol® (RDP), Citrix Independent Computing Architecture® (ICA), orVMware VMview®.

In operation 804, the remote desktop protocol server (e.g., server 120A)may determine whether a flash player (e.g., Adobe Flash Player®) isinstalled on the client computing device. The remote desktop protocolserver (e.g., server 120A) may determine whether the flash player isinstalled on the client computing device, for example, by sending amessage to the client computing device via operation of a prerequisitecheck module (e.g., prerequisite check module 325) in conjunction with anetwork module (e.g., network module 345). If the flash player is notinstalled on the client computing device, the process 800 continues toflash acceleration Subroutine A, which begins at operation 810 and isdescribed in detail below. If the flash player is installed on theclient computing device, the process 800 continues to operation 806.

In operation 806, the remote desktop protocol server determines whetherseamless or selective flash redirection settings are in effect. Theremote desktop protocol server may determine whether seamless orselective flash redirection settings are in effect via operation of theprerequisite check module (e.g., prerequisite check module 325). In oneexample, the remote desktop protocol server may determine whetherseamless or selective flash redirection settings are in effect based onthe settings stored on the server (e.g., settings 360, which includeseamless/selective flash redirection settings 365). The settings storedon the server may be entered by an administrator of the server and maybe modified by the administrator. If selective flash redirectionsettings are in effect, the process 800 continues to Selective FlashRedirection Subroutine B, which begins at operation 824 and is describedin detail below. If seamless flash redirection settings are in effect,the process 800 continues to operation 808.

In operation 808, the remote desktop protocol server renders each flashcontent item on the webpage via operation of Flash RedirectionSubroutine C for each flash content item of the webpage. FlashRedirection Subroutine C begins at operation 832 and is described indetail below.

Flash Acceleration Subroutine A

With reference to FIG. 8B, the flash acceleration Subroutine A begins atoperation 810, where the remote desktop protocol server fetches, viaoperation of the web browser (e.g., browser 350), flash data associatedwith a flash content item and passes the flash data to a flash player(e.g., flash player 355, which may correspond to Adobe Flash Player®).The flash player executes within the remote desktop protocol server. Theflash data may include, for example, a video file.

In operation 812, the remote desktop protocol server decodes the flashdata into a first raw bitmap, via operation of the flash player (e.g.,flash player 355).

In operation 813, the remote desktop protocol server captures orintercepts the decoded first raw bitmap by operation of a flashvirtualization module (e.g., server flash virtualization module 320).The flash virtualization module executes within the remote desktopprotocol server.

In operation 814, the remote desktop protocol server transmits the firstraw bitmap to an acceleration module (e.g., acceleration module 335) ofthe flash virtualization module, while foregoing transmission of thefirst raw bitmap to the client computing device.

In operation 816, the remote desktop protocol server compresses thefirst raw bitmap into an image, for example, a JPEG image. Thecompression may be completed via operation of a compression module(e.g., compression module 330) of the flash virtualization module. Insome examples, a different image file type, e.g., a GIF image, may beused in place of the JPEG image.

In operation 818, the remote desktop protocol server facilitatestransmission of the image (e.g., the JPEG image) to the client computingdevice, for example, via operation of a redirection module (e.g.,redirection module 340) or a network module (e.g., network module 345).The image may be received on the client computing device via operationof a network module (e.g., network module 430) residing within a clientflash virtualization module (e.g., client flash virtualization module420) of the client computing device.

In operation 820, the remote desktop protocol server facilitatesdecoding the image into a second raw bitmap on the client computingdevice. The decoding may be completed via a decoding module (e.g.,decoding module 425) residing within the client flash virtualizationmodule (e.g., client flash virtualization module 420) of the clientcomputing device.

In operation 822, the remote desktop protocol server facilitatesrendering of the second raw bitmap on the client computing device at alocation (e.g., on a display of the client computing device)corresponding to the flash content item. The rendering may be completedvia a rendering module (e.g., rendering module 440) residing within theclient flash virtualization module (e.g., client flash virtualizationmodule 420) of the client computing device. After operation 822, theprocess 800 ends.

One example of a process of flash acceleration 900B is illustrated inFIG. 9B and includes the steps 902B, 904B, 906B, 908B, 910B, 912B, and914B.

Selective Flash Redirection Subroutine B

With reference to FIG. 8C, the selective Flash Redirection Subroutine Bbegins at operation 824, where the remote desktop protocol serverreceives a selection of a flash content item. The selection may beentered via the web browser (e.g., browser 350) of the server executingin a remote desktop protocol client application on the client computingdevice. For example, the user of the client computing device may selecta flash content item (e.g., by clicking on the flash content item usinga mouse or touching the flash content item on a touch screen).

In operation 826, the remote desktop protocol server determines whetherthe selected flash content item is currently being redirected. Theremote desktop protocol may make this determination based on a storedstate of one or more of the web browser, the flash virtualization module(e.g., server flash virtualization module 320), the redirection module(e.g., redirection module 340) within the flash virtualization module,or the network module (e.g., network module 345) within the flashvirtualization module. Initially, all flash content item(s) on thewebpage may not be redirected (may have a state indicating that theflash content item(s) are not redirected). Alternatively, initially allflash content item(s) on the webpage may be redirected (may have a stateindicating that the flash content item(s) are redirected). Uponselection of a flash content item, the redirection state of the flashcontent item may be changed—if the flash content item is presently notredirected, the flash content item may change to being redirected; ifthe flash content item is presently being redirected, the flash contentitem may change to not being redirected. If the selected flash contentitem is not currently being redirected, the process 800 continues tooperation 828. If the selected flash content item is currently beingredirected, the process 800 continues to operation 830.

In operation 828, if the selected flash content item is not currentlybeing redirected, the remote desktop protocol server applies FlashRedirection Subroutine C for the selected flash content item. FlashRedirection Subroutine C begins at operation 832 and is described indetail below.

In operation 830, if the selected flash content item is currently beingredirected, the remote desktop protocol server terminates redirection ofthe selected flash content item and renders the selected flash contentitem at the remote desktop protocol server. The rendered raw bitmap,including the selected flash content item, is then transmitted to theclient computing device via the remote desktop protocol. The renderedraw bitmap may then be provided for display on the client computingdevice. After operation 830, the process 800 ends.

One example of a process of selective flash redirection 900C isillustrated in FIG. 9C and includes the steps 902C, 904C, 906C, 908C,910C, and 912C.

With selective flash redirection, a user may be able to select whichflash content item(s) are redirected for rendering at the clientcomputing device and which flash content item(s) are rendered on theremote desktop protocol server. As a result, use of the memory, therendering processors (e.g., processors in the central processing unit orthe graphics processing unit), or the network interface of the clientcomputing device may be reduced. In some aspects, flash content item(s)on a webpage in which the user has an interest (e.g., which the userselects) may be rendered more quickly on the client computing device,while the flash content item(s) that are not selected may be renderedmore slowly or more poorly via the remote desktop protocol connection.

Flash Redirection Subroutine C

With reference to FIG. 8D, the flash redirection Subroutine C begins atoperation 832, where the remote desktop protocol server determineswhether the client computing device is connected to the Internetdirectly (e.g., via connection 135A). For example, the remote desktopprotocol server may send a message to the client computing deviceinquiring whether the client computing device is connected to theInternet directly and receive a response from the client computingdevice (based on a determination made at the client computing device).If the client computing device is connected to the Internet directly,the process 800 continues to Internet-Based Flash Redirection SubroutineD, which begins at operation 836 and is described in detail below. Ifthe client computing device is not connected to the Internet directly,the process 800 continues to operation 834.

In operation 834, the remote desktop server determines whether theclient computing device is connected to the Internet via a proxy server(e.g., via Proxy Server 130A, via connections 145A and 155A). The proxyserver may be different from the remote desktop protocol server. Forexample, the remote desktop protocol server may send a message to theclient computing device inquiring whether the client computing device isconnected to the Internet via the proxy server and receive a responsefrom the client computing device. If the client computing device isconnected to the Internet via the proxy server, the process 800continues to Internet-Based Flash Redirection Subroutine D, which beginsat operation 836 and is described in detail below. If the clientcomputing device is not connected to the Internet (either directly orvia the proxy server), the process 800 continues to Tunnel-Based FlashRedirection Subroutine E, which begins at operation 842 and is describedin detail below.

Internet-Based Flash Redirection Subroutine D

With reference to FIG. 8E, the Internet-Based Flash RedirectionSubroutine D begins at operation 836, where the remote desktop protocolserver fetches all flash parameters of the flash content item, viaoperation of a redirection module (e.g., redirection module 340) of theflash virtualization module (e.g., server flash virtualization module320). The flash parameter(s) may include, for example, a uniformresource locator (URL) of the flash content item, a size of the flashcontent item, dimensions of the flash content item, etc. The flashparameters of the flash content item are flash parameters associatedwith the flash content item.

In operation 838, the remote desktop protocol server facilitatestransmission of the flash parameters of the flash content item to theclient computing device for feeding to the flash player (e.g., flashplayer 210A) executing on the client computing device. The flashparameters of the flash content item may include, for example, a uniformresource locator (URL) of the flash content item.

In operation 840, the remote desktop protocol server facilitatesfetching of flash data of the flash content item from a flash contentserver for playing of the flash content item via the flash playerexecuting on the client computing device. The client computing devicemay fetch the flash data of the flash content item via its Internetconnection, which may be a direct Internet connection or an Internetconnection via a proxy server. On the client computing device, the flashredirection client module (e.g. flash redirection client module 435) mayinstantiate the flash player (e.g., flash player 210A) in a position ofa display overlaying the browser of the remote desktop protocol clientapplication. After operation 840, the process 800 ends.

Tunnel-Based Flash Redirection Subroutine

With reference to FIG. 8F, the Tunnel-Based Flash Redirection SubroutineE begins at operation 842, where the remote desktop protocol serverfacilitates creation of a tunnel (e.g., tunnel 230A) between the remotedesktop protocol server and the client computing device, via operationof a tunneling module (e.g., Tunneling Module 220A) of the flashvirtualization module (e.g., server flash virtualization module 320).Upon creation of the tunnel, the remote desktop protocol server may beconnected to the client computing device via both the tunnel and theremote desktop protocol connection (e.g., connection 125A). The tunnelmay be different from the remote desktop protocol connection.

In operation 844, the remote desktop protocol server fetches all flashparameters of the flash content item, via operation of the redirectionmodule (e.g., redirection module 340) of the flash virtualizationmodule.

In operation 846, the remote desktop protocol server facilitatesfetching, via operation of the network module (e.g., Network Module 345)of the flash virtualization module, of flash data for the flash contentitem from the flash content server. The flash data may include, forexample, a flash video.

In operation 848, the remote desktop protocol server facilitatestransmission, via the tunnel, of the flash data for the flash contentitem to the client computing device for playing of the flash contentitem via the flash player (e.g., flash player 210A) executing on theclient computing device. The client computing device may receive theflash data. On the client computing device, the flash redirection clientmodule (e.g. flash redirection client module 435) may instantiate theflash player (e.g., flash player 210A) in a position of a displayoverlaying the browser of the remote desktop protocol clientapplication. The position of the display may overlay the image dataassociated with the webpage. The flash player on the client computingdevice may play the flash data in response to receipt of the flash data.After operation 848, the process 800 ends.

Flash Redirection with Scripts

FIG. 8G illustrates the operation of flash redirection in conjunctionwith the remote execution of JavaScript. At step 850, a browser (e.g.browser 350) is launched at a server (e.g. server 120A), and the browserloads a webpage having flash content. The browser may be, for example,Internet Explorer (typically using ActiveX plugins for flash support),Firefox, Chrome, or another NPAPI browser (using NPAPI plugins forflash). When using flash redirection, the browser may, at step 852, loadthe flash redirection proxy plugin (e.g. part of redirection module340). At step 854, the flash redirection proxy plugin may request theflash redirection proxy browser (e.g. part of flash redirection clientmodule 435) to load flash player 210A at the client. In this example,the flash content of the webpage depends on the execution of JavaScript,and so at step 856, flash player 210A requests the flash redirectionproxy browser at the client to execute the JavaScript. As the browsercapable of executing the JavaScript is located at the server, at step858, the flash redirection proxy browser passes the JavaScript to flashredirection proxy plugin so that it may be executed by browser 350. Atstep 859, the flash redirection proxy plugin requests browser 350 toexecute the JavaScript. At step 860, the browser at the server executesthe JavaScript and returns the result to the flash redirection proxyplugin at the server. At step 862, the flash redirection proxy pluginpasses the result of the script execution to the flash redirection proxybrowser, which then passes the result to the flash player at the client.The client, therefore, may experience playback of flash content thatinteracts with JavaScript, with the flash content rendered locally atthe client, and the JavaScript executed remotely at a server.

FIG. 8H illustrates the operation of flash redirection in conjunctionwith the remote execution of ActionScript. At step 870, a browser (e.g.browser 350) is launched at a server (e.g. server 120A), and the browserloads a webpage having flash content. The browser may be, for example,Internet Explorer (typically using ActiveX plugins for flash support),Firefox, Chrome, or another NPAPI browser (using NPAPI plugins forflash). When using flash redirection, the browser may, at step 872, loadthe flash redirection proxy plugin (e.g. part of redirection module340). At step 874, the flash redirection proxy plugin may request theflash redirection proxy browser (e.g. part of flash redirection clientmodule 435) to load flash player 210A at the client. In this example,the flash content of the webpage depends on the execution ofActionScript, and so at step 876, browser 350 requests the flashredirection proxy plugin at the server to execute the ActionScript. Asthe flash player capable of executing the ActionScript is located at theclient, at step 878, the flash redirection proxy plugin at the serverpasses the ActionScript to the flash redirection proxy browser so thatit may be executed by flash player 210A. At step 879, the flashredirection proxy browser requests flash player 210A to execute theActionScript. At step 880, the flash player at the client executes theActionScript and returns the result to the flash redirection proxybrowser at the client. At step 882, the flash redirection proxy browserpasses the result of the script execution to the flash redirection proxyplugin, which then passes the result to the browser at the server. Theclient, therefore, may experience playback of flash content thatinteracts with ActionScript, with the browser executing the ActionScriptremotely at the client.

The examples of FIGS. 8G and 8H may be applicable with eitherinternet-based flash redirection or tunnel-based flash redirection.

One example of a process 900A for presenting flash content isillustrated in FIG. 9A and includes the steps 902A, 904A, 906A, 908A,910A, 912A, 914A, 916A, 918A, 920A, 922A, 924A, 926A, 928A, 930A, 932A,934A, 936A, and 938A.

As illustrated in FIGS. 8A-8H and 9A-9C, a client or client computingdevice may be, for example, a client implementing a Wyse TCX® clientapplication (or any similar remote desktop protocol client application),and a server or remote desktop server may be, for example, a serverimplementing a Wyse TCX® server application (or any similar remotedesktop protocol server application).

Other Remarks

In one aspect, any of the clauses herein may depend from any one of theindependent clauses or any one of the dependent clauses. In one aspect,any of the clauses (e.g., dependent or independent clauses) may becombined with any other clauses (e.g., dependent or independentclauses). In one aspect, a claim may include some or all of the words(e.g., steps, operations, means or components) recited in a clause, asentence, a phrase or a paragraph. In one aspect, a claim may includesome or all of the words recited in one or more clauses, sentences,phrases or paragraphs. In one aspect, some of the words in each of theclauses, sentences, phrases or paragraphs may be removed. In one aspect,additional words or elements may be added to a clause, a sentence, aphrase or a paragraph. In one aspect, the subject technology may beimplemented without utilizing some of the components, elements,functions or operations described herein. In one aspect, the subjecttechnology may be implemented utilizing additional components, elements,functions or operations.

In one aspect, any methods, instructions, code, means, logic,components, blocks, modules and the like (e.g., software or hardware)described or claimed herein can be represented in drawings (e.g., flowcharts, block diagrams), such drawings (regardless of whether explicitlyshown or not) are expressly incorporated herein by reference, and suchdrawings (if not yet explicitly shown) can be added to the disclosurewithout constituting new matter. For brevity, some (but not necessarilyall) of the clauses/descriptions/claims are explicitly represented indrawings, but any of the clauses/descriptions/claims can be representedin drawings in a manner similar to those drawings explicitly shown. Forexample, a flow chart can be drawn for any of the clauses, sentences orclaims for a method such that each operation or step is connected to thenext operation or step by an arrow. In another example, a block diagramcan be drawn for any of the clauses, sentences or claims havingmeans-for elements (e.g., means for performing an action) such that eachmeans-for element can be represented as a module for element (e.g., amodule for performing an action).

Those of skill in the art would appreciate that items such as thevarious illustrative blocks, modules, elements, components, methods,operations, steps, and algorithms described herein (e.g., the clientcomputing device 110A, the server 120A, the proxy server 130A, the flashcontent server 150A, and the components therein) may be implemented ashardware, computer software, or a combination of both.

To illustrate the interchangeability of hardware and software, itemssuch as the various illustrative blocks, modules, elements, components,methods, operations, steps, and algorithms have been described generallyin terms of their functionality. Whether such functionality isimplemented as hardware or software depends upon the particularapplication and design constraints imposed on the overall system.Skilled artisans may implement the described functionality in varyingways for each particular application.

In one aspect, “means,” a block, a module, an element, a component or aprocessor may be an item (e.g., one or more of blocks, modules,elements, components or processors) for performing one or more functionsor operations. In one aspect, such an item may be an apparatus,hardware, or a portion thereof. In one example, an item may have astructure in the form of, for example, an instruction(s) for performingthe function(s) or operation(s), where the instruction(s) are encoded orstored on a machine-readable medium, on another device, or on a portionthereof, where an instruction(s) may be software, an application(s), asubroutine(s), or a portion thereof. In an example, an item may beimplemented as one or more circuits configured to perform thefunction(s) or operation(s). A circuit may include one or more circuitsand/or logic. A circuit may be analog and/or digital. A circuit may beelectrical and/or optical. A circuit may include transistors. In anexample, one or more items may be implemented as a processing system(e.g., a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA),etc.). Those skilled in the art will recognize how to implement theinstructions, circuits, and processing systems.

A reference to an element in the singular is not intended to mean “oneand only one” unless specifically so stated, but rather “one or more.”For example, a server may refer to one or more servers, a tunnel mayrefer to one or more tunnels, a connection may refer to one or moreconnections, etc.

Unless specifically stated otherwise, the term “some” refers to one ormore. Pronouns in the masculine (e.g., his) include the feminine andneuter gender (e.g., her and its) and vice versa. Headings andsubheadings, if any, are used for convenience only and do not limit theinvention.

The word “exemplary” is used herein to mean “serving as an example orillustration.” Any aspect or design described herein as “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs. In one aspect, various alternative configurationsand operations described herein may be considered to be at leastequivalent.

A phrase such as an “aspect” does not imply that such aspect isessential to the subject technology or that such aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations.An aspect may provide one or more examples. A phrase such as an aspectmay refer to one or more aspects and vice versa. A phrase such as an“embodiment” does not imply that such embodiment is essential to thesubject technology or that such embodiment applies to all configurationsof the subject technology. A disclosure relating to an embodiment mayapply to all embodiments, or one or more embodiments. An embodiment mayprovide one or more examples. A phrase such an embodiment may refer toone or more embodiments and vice versa. A phrase such as a“configuration” does not imply that such configuration is essential tothe subject technology or that such configuration applies to allconfigurations of the subject technology. A disclosure relating to aconfiguration may apply to all configurations, or one or moreconfigurations. A configuration may provide one or more examples. Aphrase such a configuration may refer to one or more configurations andvice versa.

In one aspect of the disclosure, when actions or functions are describedas being performed by an item (e.g., receiving, determining, fetching,creating, providing, generating, converting, displaying, playing,notifying, accepting, selecting, controlling, transmitting, reporting,sending, or any other action or function), it is understood that suchactions or functions may be performed by the item directly orindirectly. In one aspect, when a module is described as performing anaction, the module may be understood to perform the action directly. Inone aspect, when a module is described as performing an action, themodule may be understood to perform the action indirectly, for example,by facilitating, enabling or causing such an action.

In one aspect, unless otherwise stated, all measurements, values,ratings, positions, magnitudes, sizes, and other specifications that areset forth in this specification, including in the claims that follow,are approximate, not exact. In one aspect, they are intended to have areasonable range that is consistent with the functions to which theyrelate and with what is customary in the art to which they pertain.

In one aspect, the term “coupled” or the like may refer to beingdirectly coupled. In another aspect, the term “coupled” or the like mayrefer to being indirectly coupled.

Various items may be arranged differently (e.g., arranged in a differentorder, or partitioned in a different way) all without departing from thescope of the subject technology. In one example, the subject technologymay be implemented without use of the proxy server 130A or without useof the tunnel 230A. In one aspect of the disclosure, the elementsrecited in the accompanying claims may be performed by one or moremodules or sub-modules.

It is understood that the specific order or hierarchy of steps,operations or processes disclosed is an illustration of exemplaryapproaches. Based upon design preferences, it is understood that thespecific order or hierarchy of steps, operations or processes may berearranged. Some of the steps, operations or processes may be performedsimultaneously. Some or all of the steps, operations, or processes maybe performed automatically, without the intervention of a user. Theaccompanying method claims, if any, present elements of the varioussteps, operations or processes in a sample order, and are not meant tobe limited to the specific order or hierarchy presented.

The disclosure is provided to enable any person skilled in the art topractice the various aspects described herein. The disclosure providesvarious examples of the subject technology, and the subject technologyis not limited to these examples. Various modifications to these aspectswill be readily apparent to those skilled in the art, and the genericprinciples defined herein may be applied to other aspects.

All structural and functional equivalents to the elements of the variousaspects described throughout this disclosure that are known or latercome to be known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe claims. Moreover, nothing disclosed herein is intended to bededicated to the public regardless of whether such disclosure isexplicitly recited in the claims. No claim element is to be construedunder the provisions of 35 U.S.C. §112, sixth paragraph, unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” Furthermore, to the extent that the term “include,” “have,” or thelike is used, such term is intended to be inclusive in a manner similarto the term “comprise” as “comprise” is interpreted when employed as atransitional word in a claim.

The Title, Background, Summary, Brief Description of the Drawings andAbstract of the disclosure are hereby incorporated into the disclosureand are provided as illustrative examples of the disclosure, not asrestrictive descriptions. It is submitted with the understanding thatthey will not be used to limit the scope or meaning of the claims. Inaddition, in the Detailed Description, it can be seen that thedescription provides illustrative examples and the various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed subject matter requires morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed configuration or operation. The followingclaims are hereby incorporated into the Detailed Description, with eachclaim standing on its own as a separately claimed subject matter.

The claims are not intended to be limited to the aspects describedherein, but is to be accorded the full scope consistent with thelanguage claims and to encompass all legal equivalents. Notwithstanding,none of the claims are intended to embrace subject matter that fails tosatisfy the requirement of 35 U.S.C. §101, 102, or 103, nor should theybe interpreted in such a way. Any unintended embracement of such subjectmatter is hereby disclaimed.

Herein, a computer-readable non-transitory storage medium or media mayinclude one or more semiconductor-based or other integrated circuits(ICs) (such, as for example, field-programmable gate arrays (FPGAs) orapplication-specific ICs (ASICs)), hard disk drives (HDDs), hybrid harddrives (HHDs), optical discs, optical disc drives (ODDs),magneto-optical discs, magneto-optical drives, floppy diskettes, floppydisk drives (FDDs), magnetic tapes, solid-state drives (SSDs),RAM-drives, SECURE DIGITAL cards or drives, any other suitablecomputer-readable non-transitory storage media, or any suitablecombination of two or more of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

This scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsherein that a person having ordinary skill in the art would comprehend.The scope of this disclosure is not limited to the example embodimentsdescribed or illustrated herein. Moreover, although this disclosuredescribes or illustrates respective embodiments herein as includingparticular components, elements, functions, operations, or steps, any ofthese embodiments may include any combination or permutation of any ofthe components, elements, functions, operations, or steps described orillustrated anywhere herein that a person having ordinary skill in theart would comprehend. Furthermore, reference in the appended claims toan apparatus or system or a component of an apparatus or system beingadapted to, arranged to, capable of, configured to, enabled to, operableto, or operative to perform a particular function encompasses thatapparatus, system, component, whether or not it or that particularfunction is activated, turned on, or unlocked, as long as thatapparatus, system, or component is so adapted, arranged, capable,configured, enabled, operable, or operative.

What is claimed is:
 1. A method at a server comprising: loading aredirection module, wherein the redirection module comprises a flashredirection proxy plugin to support functionality of a flash player at aclient; loading in a browser a webpage comprising flash content; in thebrowser, loading the flash redirection proxy plugin, wherein the flashredirection proxy plugin redirects one or more flash parameters from thebrowser to the flash player; by the flash redirection proxy plugin,sending a request to a flash redirection proxy browser at the client toload the flash player at the client; by the browser, requesting theflash redirection proxy plugin to execute an ActionScript; by the flashredirection proxy plugin, passing the ActionScript to the flashredirection proxy browser at the client; receiving by the flashredirection proxy plugin from the flash redirection proxy browser aresult of executing the ActionScript; and providing, by the flashredirection proxy plugin, the result to the browser.
 2. The method ofclaim 1 further comprising: determining based, at least in part, on astored state of one or more of the browser, a flash virtualizationmodule, the redirection module, and a network module if the flashcontent is currently being redirected.
 3. The method of claim 1, whereinthe client is remote from the server.
 4. The method of claim 1, whereinthe flash content comprises a plurality of flash content items of thewebpage.
 5. The method of claim 1, wherein the flash content interactswith the ActionScript.
 6. The method of claim 1 further comprising:fetching the one or more flash parameters from the flash content via theredirection module, wherein the one or more flash parameters compriseone or more of a uniform resource locator, a size, and dimensions.
 7. Amethod comprising: receiving at a flash redirection proxy browser of aclient a request from a server to load a flash player at the client; bythe flash player, requesting the flash redirection proxy browser toexecute a JavaScript; by the flash redirection proxy browser, passingthe JavaScript to a flash redirection proxy plugin at the server;receiving by the flash redirection proxy browser from the flashredirection proxy plugin a result of executing the JavaScript; providingthe result to the flash player; and rendering locally at the clientreceived flash content, wherein the result comprises the flash content.8. The method of claim 7, wherein the flash content is received during aremote desktop protocol session.
 9. The method of claim 7, wherein theflash content comprises a plurality of flash content items of thewebpage.
 10. The method of claim 7, wherein the flash content interactswith the JavaScript.
 11. The method of claim 7, wherein the server isremote from the client.
 12. The method of claim 7, wherein the flashplayer receives the flash content via a tunnel connection.
 13. Themethod of claim 7, wherein a flash redirection client moduleinstantiates the flash player.
 14. One or more computer-readablenon-transitory storage media of a server embodying software that, whenexecuted: loads a redirection module, wherein the redirection modulecomprises a flash content redirection proxy plugin to supportfunctionality of a flash player at a client; loads in a browser awebpage comprising the flash content; loads the flash redirection proxyplugin, wherein the flash redirection proxy plugin redirects one or moreflash parameters from the browser to the flash player; sends a requestto a flash redirection proxy browser at the client by the flashredirection proxy plugin to load the flash player at the client;requests by the browser to the flash redirection proxy plugin to executean ActionScript; passes the ActionScript by the flash redirection proxyplugin to the flash redirection proxy browser at the client; receives bythe flash redirection proxy plugin from the flash redirection proxybrowser a result of executing the ActionScript; and provides the resultto the browser.
 15. The media of claim 14, wherein the software furtherwhen executed: determines based, at least in part, on a stored state ofone or more of the browser, a flash virtualization module, theredirection module, and a network module if the flash content iscurrently being redirected.
 16. The media of claim 14, wherein theserver is remote from the client.
 17. The media of claim 14, wherein theflash content comprises a plurality of flash content items of thewebpage.
 18. The media of claim 14, wherein the flash content interactswith the ActionScript.
 19. The media of claim 14, wherein the softwarefurther when executed: fetches the one or more flash parameters from theflash content via the redirection module, wherein the one or more flashparameters comprise one or more of a uniform resource locator, a size,and dimensions.
 20. The media of claim 14, wherein the result is sent tothe flash redirection proxy browser during a remote desktop protocolsession.